[Feature] Support ScanNet semantic segmentation dataset #390

Wuziyi616 · 2021-03-29T09:33:04Z

Add ScanNet dataset for semantic segmentation task, including data pre-processing, data loading for train and test, evaluation metrics and result visualization.

codecov · 2021-03-29T09:51:14Z

Codecov Report

Merging #390 (91e6e1f) into master (d055876) will increase coverage by 0.97%.
The diff coverage is 76.17%.

❗ Current head 91e6e1f differs from pull request most recent head dd0f856. Consider uploading reports for the commit dd0f856 to get more accurate results

@@            Coverage Diff             @@
##           master     #390      +/-   ##
==========================================
+ Coverage   49.73%   50.70%   +0.97%     
==========================================
  Files         181      182       +1     
  Lines       12445    12765     +320     
  Branches     1950     2014      +64     
==========================================
+ Hits         6189     6473     +284     
- Misses       5867     5875       +8     
- Partials      389      417      +28

Flag	Coverage Δ
unittests	`50.70% <76.17%> (+0.97%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmdet3d/datasets/pipelines/__init__.py	`100.00% <ø> (ø)`
mmdet3d/datasets/pipelines/formating.py	`54.73% <0.00%> (ø)`
mmdet3d/core/evaluation/seg_eval.py	`16.32% <12.50%> (-1.86%)`	⬇️
mmdet3d/core/visualizer/show_result.py	`75.00% <66.66%> (-4.67%)`	⬇️
mmdet3d/datasets/custom_3d_seg.py	`67.14% <67.14%> (ø)`
mmdet3d/datasets/pipelines/transforms_3d.py	`84.93% <82.95%> (+0.59%)`	⬆️
mmdet3d/core/points/base_points.py	`80.92% <88.88%> (+15.01%)`	⬆️
mmdet3d/datasets/pipelines/loading.py	`83.13% <89.47%> (+8.96%)`	⬆️
mmdet3d/core/visualizer/__init__.py	`100.00% <100.00%> (ø)`
mmdet3d/datasets/__init__.py	`100.00% <100.00%> (ø)`
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d055876...dd0f856. Read the comment docs.

Wuziyi616 · 2021-03-29T10:18:57Z

In general, I did five things in this PR.

I modify the ScanNet pre-processing files used in the point cloud detection task. I remove the max_num_point subsampling in preprocessing.
I write data loading pipelines for train and test data generation. The core class is IndoorPatchPointSample where we perform sampling to get NUM_POINTS points within a patch (usually 1.5mx1.5m) as one entry. In training, we randomly sample this patch, while in testing, we use the sliding window strategy. Also, I modify some functions in BasePoint to accommodate the pipelines.
I write a dataset class for semantic segmentation. I think it's quite different from the detection dataset, because we need to support room re-sampling and label re-weighting. I also write a ScanNet segmentation config file based on its detection counterpart.
I modify the segmentation evaluation function. In the indoor scene segmentation task, there are some unannotated points, marked by ignore_index in the code. So we need to filter them in loss calculation & evaluation metrics & visualizations.
I write a visualization function for semantic segmentation.

mmdet3d/core/evaluation/seg_eval.py

data/scannet/batch_load_scannet_data.py

mmdet3d/core/evaluation/seg_eval.py

tests/test_data/test_pipelines/test_indoor_sample.py

mmdet3d/datasets/scannet_dataset.py

mmdet3d/datasets/pipelines/transforms_3d.py

configs/_base_/datasets/scannet_seg-3d-20class.py

data/scannet/batch_load_scannet_data.py

mmdet3d/core/evaluation/seg_eval.py

mmdet3d/core/visualizer/show_result.py

mmdet3d/datasets/scannet_dataset.py

mmdet3d/datasets/pipelines/transforms_3d.py

Wuziyi616 · 2021-03-30T13:24:35Z

I modify the code according to all the comments (the last 9 commits). I merge the code with master branch using rebase because there are conflicts in the code.

mmdet3d/datasets/pipelines/transforms_3d.py

mmdet3d/core/evaluation/seg_eval.py

…pping

…a loading and add unittest

…or loading

…o unittest

…rocessing

…taset inheriting from base dataset

Wuziyi616 · 2021-04-06T06:39:47Z

Merge with master branch. All comments cleared.

mmdet3d/datasets/pipelines/transforms_3d.py

tests/test_utils/test_points.py

tools/data_converter/scannet_data_utils.py

tools/data_converter/indoor_converter.py

tests/test_data/test_datasets/test_scannet_dataset.py

* add merge_shape_concate * add some peephole optimization * bug fixing * fix for torch1.9 * add flatten cls head * add subgraph matcher with attribute * add ut * fix lint * remove onnx2ncnn * add opset version * axis name * fix peephole * fix symbol compare * add docs

Wuziyi616 requested review from yezhen17 and ZwwWayne March 29, 2021 09:33

yezhen17 reviewed Mar 29, 2021

View reviewed changes

ZwwWayne reviewed Mar 30, 2021

View reviewed changes

Tai-Wang reviewed Mar 30, 2021

View reviewed changes

mmdet3d/datasets/pipelines/transforms_3d.py Outdated Show resolved Hide resolved

Tai-Wang reviewed Mar 30, 2021

View reviewed changes

mmdet3d/datasets/pipelines/transforms_3d.py Outdated Show resolved Hide resolved

Wuziyi616 force-pushed the scannet_semseg_dataset branch from 07e38e4 to 15d977b Compare March 30, 2021 13:20

yezhen17 approved these changes Mar 31, 2021

View reviewed changes

mmdet3d/datasets/pipelines/transforms_3d.py Outdated Show resolved Hide resolved

mmdet3d/datasets/pipelines/transforms_3d.py Show resolved Hide resolved

mmdet3d/datasets/pipelines/transforms_3d.py Outdated Show resolved Hide resolved

Wuziyi616 requested review from ZwwWayne and Tai-Wang March 31, 2021 03:07

Tai-Wang reviewed Apr 1, 2021

View reviewed changes

mmdet3d/core/evaluation/seg_eval.py Outdated Show resolved Hide resolved

Tai-Wang approved these changes Apr 1, 2021

View reviewed changes

Wuziyi616 force-pushed the scannet_semseg_dataset branch 3 times, most recently from c0440e4 to 698a1de Compare April 2, 2021 15:41

Wuziyi616 added 12 commits April 6, 2021 14:36

remove max_num_point in ScanNet data preprocessing

85c15a4

add config file for ScanNet semantic segmentation dataset

5d9aca6

modify NormalizePointsColor in pipeline

f4bbb00

add visualization function for semantic segmentation

2adea6a

add ignore_index to semantic segmentation visualization function

4412691

add ignore_index to semantic segmentation evaluation function

d8ecd43

fix ignore_index bug in semantic segmentation evaluation function

3490ca6

add test function to check ignore_index assignment in PointSegClassMa…

fea1b62

…pping

fix slicing bug in BasePoints class and add unittest

dd7df61

add IndoorPatchPointSample class for indoor semantic segmentation dat…

e721eb1

…a loading and add unittest

modify LoadPointsFromFile class and its unittest to support point col…

943492b

…or loading

fix data path in unittest

b680f71

Wuziyi616 added 14 commits April 6, 2021 14:36

fix small bugs & change variable names for clarity & add more cases t…

417101c

…o unittest

move room index resampling and label weight computation to data pre-p…

1705ada

…rocessing

add option allowing user to determine whether to sub-sample point clouds

f9b975f

fix typos & change .format to f-string & fix link in comment

46e0133

save all visualizations into .obj format for consistency

deda71d

infer num_classes from label2cat in eval_seg function

ea670db

add pre-computed room index and label weight for ScanNet dataset

5ae21c5

replace .ply with .obj in unittests and documents

b862eba

add TODO in case data is on ceph

3591efa

add base dataset for all semantic segmentation tasks & add ScanNet da…

d11dc79

…taset inheriting from base dataset

rename class for consistency

e0604af

fix minor typos in comment

996afd8

move Custom3DSegDataset to a new file

8940bbf

modify BasePoint setter function to enable attribute adding

35bfb35

Wuziyi616 force-pushed the scannet_semseg_dataset branch from 698a1de to 35bfb35 Compare April 6, 2021 06:37

ZwwWayne reviewed Apr 6, 2021

View reviewed changes

Wuziyi616 added 4 commits April 6, 2021 16:06

add unittest for NormalizePointsColor and fix small bugs

c4c148d

fix unittest for BasePoints

bd06d3f

modify ScanNet data pre-processing scripts

253bf53

change ignore_idx to -1 in seg_eval function

460ed04

ZwwWayne reviewed Apr 6, 2021

View reviewed changes

tests/test_data/test_datasets/test_scannet_dataset.py Outdated Show resolved Hide resolved

remove sliding inference from PatchSample function and modify unittest

71ff97b

Wuziyi616 requested a review from ZwwWayne April 6, 2021 11:44

remove PatchSample from scannet seg test_pipeline

dd0f856

ZwwWayne approved these changes Apr 7, 2021

View reviewed changes

ZwwWayne merged commit 9cb75e7 into open-mmlab:master Apr 7, 2021

Wuziyi616 deleted the scannet_semseg_dataset branch April 7, 2021 07:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature] Support ScanNet semantic segmentation dataset #390

[Feature] Support ScanNet semantic segmentation dataset #390

Wuziyi616 commented Mar 29, 2021

codecov bot commented Mar 29, 2021 •

edited

Loading

Wuziyi616 commented Mar 29, 2021

Wuziyi616 commented Mar 30, 2021

Wuziyi616 commented Apr 6, 2021

[Feature] Support ScanNet semantic segmentation dataset #390

[Feature] Support ScanNet semantic segmentation dataset #390

Conversation

Wuziyi616 commented Mar 29, 2021

codecov bot commented Mar 29, 2021 • edited Loading

Codecov Report

Wuziyi616 commented Mar 29, 2021

Wuziyi616 commented Mar 30, 2021

Wuziyi616 commented Apr 6, 2021

codecov bot commented Mar 29, 2021 •

edited

Loading